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We compute the probability of positive large deviations of the free energy per spin in mean- 
field Spin-Glass models. The probability vanishes in the thermodynamic limit as P{Af) oc 
exp[-iV^L2(A/)]. For the Sherrington-Kirkpatrick model we find L2(A/) = 0(A/)^^''^ in good 
agreement with numerical data and with the assumption that typical small deviations of the free 
energy scale as N^'''^ . For the spherical model we find L2(A/) = 0(A/)'^ in agreement with recent 
findings on the fluctuations of the largest eigenvalue of random Gaussian matrices. The computa- 
tion is based on a loop expansion in replica space and the non-gaussian behaviour follows in both 
cases from the fact that the expansion is divergent at all orders. The factors of the leading order 
terms are obtained resumming appropriately the loop expansion and display universality, pointing 
to the existence of a single universal distribution describing the small deviations of any model in 
the full-Replica-Symmetry-Breaking class. 



The free energy density of a random system model fluc- 
tuates over the disorder with a distribution that in the 
thermodynamic limit becomes peaked around the typi- 
cal value ftyp. The probability of observing 0(1) devi- 
ations (i.e. large deviations) from the typical value are 
exponentially small. In some simple systems (like the 
mean field spherical model for spin glasses) the proba- 
bility of large deviations is highly asymmetric In 
the case of negative deviations A/ — f — ftyp we have 
P(A/) (X exp[— 7VL(A/)] [2]; for positive deviations the 
probability is much smaller: P{Af) cx exp[— N'^ L2{Af)]. 

This unusual asymmetric behaviour related to the dis- 
tribution the large deviations of the lowest eigenvalue 
of a large Gaussian random matrix. The probability 
of observing large positive deviations scales indeed as 
exp[— 0(iV^)] and had been recently computed in On 
the other hand the typical small deviation distribution of 
the lowest eigenvalue are described by a universal func- 
tion in the large A'^-limit that was computed by Tracy and 
Widom 3 and has since appeared in many apparently 
unrelated problems (see e.g. [l|). 

In this letter we show that in the mean-field 
Sherrington-Kirkpatrick(SK) spin-glass model a simi- 
lar scaling is present and the probability distribution 
L2(A/) can be computed using the hierarchical replica- 
symmetry-breaking(RSB) ansatz (the function L(A/) 
that is relevant for negative deviations was computed in 
0). Interestingly enough our results also show a great 
deal of universality suggesting that the small deviations 
of the free energy of the whole class of full-KSB models 
are also described by the same universal function at any 
temperature in the spin-glass phase. 

The computation of L2(A/) is much more complex 
than that of L(A/): indeed the function L{Af) is infi- 
nite for positive A/ [3, f5| . If we attack the problem in 
the replica framework it leads to an action of a matrix 
Qab of size n X n where n — aN with negative a. Usu- 
ally fluctuations around the saddle point are negligible 
(they give a contribution to the total free energy that is 
proportional to n^) however they play a crucial role in 
this case because the number of elements of the matrix 



Q is 0{N'^). We have also to face the fact the the saddle 
point is marginal (i.e. there are zero modes that lead 
to divergent fluctuations) thus if we proceed naively we 
obtain a series divergent at all orders. The marginality 
of mean-field theory is a general well-known feature of 
fuU-RSB spin-glass and it stands as the main difficulty 
to solve many of the open problems in the field. In par- 
ticular it leads to non-trivial finite-size correction in the 
SK model for which no systematic computation scheme 
has been devised up to now Besides the fact that 
no theory can be considered satisfactory if it does not 
allow to include deviations and corrections, these non- 
trivial mean-field exponents have been argued to be rel- 
evant also for finite-dimensional quantities, notably the 
stiffness exponent 0- Furthermore we do not know how 
to treat loop corrections to the theory in the spin-glass 
phase below six dimensions Q. Thus the need to tackle 
the marginal nature of the theory makes the computation 
as interesting as the problem itself. 

In order to compute the so called sample complexity 
L2(A/) = — ln(P(A/))/7V^ we consider the average par- 
tition function of n = aN replicas: 

$(a)^-^lnZ^, (1) 

with a negative (the bar denotes the average over the dis- 
order). In the usual approach, when we need to compute 
the typical free energy {ftyp) the number (n) of replicas 
goes to zero, but in this case it has to go to —oo. 

The functional $(q;) is the Legendre transform of 
L2(A/). The latter can be obtained as —L2{Af) = 
ctPftyp + ct/3A/ — $(a) where a is the solution of (3 ftyp + 
/3A/ — d^{a)/da. The main result of this letter is the 
evaluation of $(0;) for small a. 

Let us now consider what happens in the SK spin- 
glass model where H = k Ji,k(yi<Jk with the u's be- 
ing Ising spins and the J's being random Gaussian vari- 
able with zero average and variance 1/N . Through stan- 
dard manipulations we rewrite ([1]) as an the integral of 
an action depending on a symmetric matrix Qab with 
a,b ~ 1, . . . , olN and Qaa = 0. As the size of the matrix 
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is 0{N) we cannot simply take the saddle point and we 
have consider the integral of 0{N'^) elements of Qat- In 
order to perform the integration we find convenient to 
divide the matrix Q^b in {^^) blocks of size nxn, with 
n some parameter between 1 and aN that will be eventu- 
ally sent to zero. The generic matrix element will be writ- 
ten as Q^^fj where the upper indices i, j — 1, . . . , Na/n la- 
bel different blocks and the lower indices a,b ^ 1, . . . ,n 
label elements inside block ij. 

The resulting action admits a saddle point with van- 
ishing off-diagonal blocks Q'-' — for all i ^ j and we 
integrate out the elements of the off-diagonal blocks i ^ j 
around their zero saddle-point value. After the integra- 
tion we are left with an integral over the diagonal blocks 
of the exponential of a 0{N'^) action over which we will 
take the saddle point. We will consider here the sim- 
plest situation where the saddle points are such that all 
the blocks on the diagonal are equal to a given nxn 
hierarchical matrix Q, i.e. = Q Vi. At the end the 



function will be obtained as the saddle-point value 
of the following functional over the nxn diagonal block 
Q: 

^{a,Q) = a(3F[Q] + S[Q,a] (2) 

where F[Q] is the standard SK free energy functional of 
a, n X n matrix Qab that is zero on the diagonal: 

a<b {s} a<b 

. (3) 

and S[Q, a] is the contribution of the fluctuations. Now 
S[Q,a] is small for small a and it should be treated as a 
perturbation: in order to obtain the first non trivial term 
we should compute S'[(3,a] at the saddle point of F[Q]. 
This is not a easy task: indeed we find that S'[(5, a] can 
be written as: 
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where Aq with a — 1, . . . ,n are the eigenvalues of the 
matrix Pab = (saSb) and the averages (•) are computed 
with respect to a single diagonal block. In particular if 
Qab extremizes F[Q] we have Qab = (saSb) and there- 
fore Pab = Sab + Qab- Thc abovc expression is valid at 
the third order in Q^-'j^ which is enough to get the first 
non-linear term in <i>(Q!). The diagonal structure above 
has been obtained performing various manipulations on 
the original integral. In particular the matrix P enters 
only though its eigenvalues because the relevant expres- 
sions are rotationally invariant at the order considered 
and the integral in S[Q, a] is invariant under a simulta- 
neous rotation of all the blocks Q*-' . The parameter p has 
been introduced for later convenience and has to be put 
to zero eventually (it has thc physical meaning of adding 
a small perturbation on thc couplings of replicas in differ- 
ent blocks effectively removing the degeneracy of permu- 
tations among them). Note that in the previous expres- 
sion the off-diagonal indices run from 1 to ~N/n because 
we have exploited the fact that the function a] in 
the thermodynamic limit is invariant under a rescaling 
{a -fba.N -> bN). 

The above expression is suitable for an expansion in 
powers of a. The first 0{o?) term is obtained performing 
the Gaussian integral: 
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S[Q, «] = £^ E 1^(1 - P^^-^b + + O(a') (5) 

ah 



In the limit a — s- the extremum of tf>[Q, a] is given by 
the extremum of F[Q\ with the first 0{a^) correction ([5]) 
evaluated on the free solution where Pab = Sab + Qab- In 
the 7T, — !■ limit the Gaussian correction turns out to be 
divergent if p = 0, this is because the lowest eigenvalue 
Ao of Pab obeys the following relationship: Aq = 1 — 
fn l{x)dx — 1/(3 that holds exactly at all temperatures 
9]. The divergence of the 0{a^) correction suggests that 
the first non- linear term in $(q;) has a power smaller than 
two. We also expect that the series in powers of a will 
be divergent at all orders and we will have to resum it in 
some way. 

Before going into the next step of the computation, we 
consider a similar treatment of the spherical model. At 
zero temperature the energy is minus the largest eigen- 
value of a random Gaussian N x N matrix. The log- 
arithm of the probability that the largest eigenvalue is 
lower than its typical value is indeed 0(7V^) and has 
been recently computed These results tell us that 
in the zero-temperature limit we have: $(a) = —a (3 — 
1 1 a (3\i +o{a^^'^). Repeating the above procedure we ob- 
tain that for the spherical model $(a) can be computed 
as the saddle-point value of a functional similar to 
The first 0{a) term depends on a symmetric nxn matrix 
Qab (not vanishing on the diagonal) and on a parameter 
z enforcing the spherical constraint: 

m^H£EQ^.-i + ^Trln[./-fQ] (6) 

ab 
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where / is the identity matrix. The second term is equal 
(at the third order in Q^^) to expression ^ provided 
we have (in matrix notation) P — {2zl — 0^Q)~^ . The 
solution of the spherical model in the a — > limit is given 
by a replica-symmetric (RS) nx n matrix Qab = q — l—T 
UM and again the lowest eigenvalue Ao of P turns out to 
obey the relationship Aq = leading to a divergent 
O(a^) correction. 

We now face the task of computing the leading correc- 
tions that diverges in the p ^ limit. This can be done 
by studying the diagrammatic loop expansion oi S[Q,a]. 
We introduce capital indices that corresponds to a cou- 
ple of upper and lower indices A = (^). Given a graph 
with vertices of degree three we have to associate to each 
line in it a couple of different capital indices in such a 
way that each vertex of the graph corresponds to a term 
in the action of eq. (|4]). This limits the number of K 
free indices and for a graphs with L loops one can prove 
that K < L + 1. Valid indexed graphs Gj at four loops 
are represented in fig. [TJ For a graph G/ with V ver- 




structure of the propagator. In particular given an in- 
dexed graph each line with lower indices ab in the graph 
corresponds to a factor /3^AaA6/(p-|-l — /3^AaA(,). The re- 
sulting object has to be summed over the K free indices 
(each running from 1 to n) leading to: 
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(8) 



where we have borrowed the factor from ([7]). To 

perform this summation we must use some well know 
results on the eigenvalues of a replica symmetric matrix Q 
and a hierarchical matrix Qab characterized by a function 
q{x) [H. 

We consider the first RS case: after some algebra we 
find that at the leading order in p the final result at 
leading order in p is: 
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1 
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where / is the number of lines of the graph. 

In the full RSB case it turns out that the dominant 
divergent contribution comes from shape of the function 
q around x = that we assume to be linear. At the 
end of the day we obtain that at leading order in p the 
contribution of the diagram is: 
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FIG. 1: Indexed Graphs of the cubic theory up to four loops. 
Note that each index must belong to a closed path on the 
graph. The first six graphs are relevant because they satisfy 
the condition K = L + 1 while graph (g) is irrelevant be- 
cause K = 4<L+1 — 5 and therefore it yields an 0{N~^) 
contribution. 



tices we have a factor ~{~a/N)^^^^^ /V\ times a factor 
/3^AaA6/(p -I- 1 — j3'^\a\b) where ab are the lower indices 
on each line. We sum over all the free K upper indices 
(they can take values 1, . . . , —N/n) and vertex permu- 
tations of the graph. This yields a factor V\{—N/n)^ 
that has to be divided by an appropriate symmetry fac- 
tor M{Gi) to avoid overcounting. Relevant graphs are 
those that yield an 0{N'^) contribution, i.e. they satisfy 
the condition K = V/2 + 2 or equivalently K = L + 1. 
Consistently it can be shown that no graph can yield a 
contribution greater than 0{N^). Summarizing the sum 
over the upper indices yields the factor: 



(-a)^(-l)^^+i 



M{Gi)n 



K 



(7) 



that multiplies the result coming from the sum over the 
lower indices that we analyze now. In order to sum over 
the lower indices we have to take care of the non trivial 



where we used the operator Ax[g] = — /q°° ^^d,^- 

As we already noticed a relevant diagram has K = L+1 
therefore in the cubic theory we have / = iK — Q. Re scal- 
ing the variable p respectively as p = {— a(3 T q{Q) / z)^ , 
p — {—a/Sq/ z)i and multiplying by the factors com- 
ing from ([7]) we get an expression of S[Q, a] in terms 
of two functions fB.s{z) and ffRssiz). The loop expan- 
sion yields the scries of the two functions in powers of z 
. Taking the z oo limit we eventually obtain: 



S[Q,a] = {-al3^Tmy^^'CfRSB+ o{-a)^^/' 
S[Q,a] = {-al3qf'^CB^s+o{-af'^ (11) 

where GfRSB = lim^^oo ^;^//flss(2) and Grs = 
lim2_>oo z^ fRs{z). At fourth loop order the diagrams are 
shown in fig. ^ and lead to the following expressions: 



Irsb{z) = -- + 0.456 z- 3.278 z^-f 36.11 z3 + 0(a;'') 
fRs{z) - 



-^ + ^z-19z^ + 443 z^ + 0{z*) 



The zero-th order terms comes from a similar treatment 
of the Gaussian contribution ([5]). The problem of ex- 
tracting the z — > oo behaviour of functions like fRssiz) 
and fRs{z) from their expansions in powers of z has 
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FIG. 2: Numerical Sample Complexity vs. Energy difference 
at zero temperature for the SK model (data of Ref. [ij cour- 
tesy of the Authors) . The data are rescaled with the exponent 
5/12 and are well fitted by a linear function —.765 Ae, which 
combined with the result limT-.o Tq{0) — .743368 leads to 
CfRSB = -.64(2) 



already appeared in spin-glass literature [l^ in a simi- 
lar context and various resummation methods have been 
devised. Performing similar analysis on the above se- 
ries we obtained the following estimates: Crs — —.68, 
CfRSB = —.66 with a 15% error. The estimate for Crs 
is in good agreement with the exact result Crs = — 2/3 
quoted above. 

In the fRSB case we obtain from ([Tl]) : 

^ lnP(A/) = -a+[Tg(0)]-«/^ A/12/5 + o(A/i2/5) 

(12) 

Where a+ = (-C/iisal,^/^ 35(7/3)2/5/(24/5144). The 
numerical data of Ref. [ij at zero temperature are highly 
consistent with the 12/5 scaling, see fig. a linear fit 
on the data corresponding to = 150 combined with 
\imT^oTq{0) — .743 [2| leads to an estimate Cjrsb — 
-.64(2). 

Remarkably the previous results display a great deal of 



universality. Indeed the exponents 12/7 and 3/2 and the 
coefficients Cjhsb and Crs depends only on the fRSB 
or RS structure of the eigenvalues of the matrix P. The 
sole dependence on the actual model and on the temper- 
ature being respectively through the parameters y^TqiO) 
and q. Interestingly enough the same universal behaviour 
is displayed by the function L(A/), again both for RS 
and fRSB 2]. For both i2(A/) and L{Af) we expect the 
universal behaviour to hold only at the leading order for 
small A/, nevertheless this points towards universality of 
the corresponding small-deviations distribution. Indeed 
the reader may have already noticed that the 12/5 expo- 
nent in (|12[) is fully consistent with the assumptions that 
the large positive deviations match the far right tail of 
the distribution of the small-deviations of the free energy 
density if they scales as / — /at = 0{N-^^^) [l^, where 
/jv is the sample average at size N. At present the latter 
hypothesis is widely accepted in the literature after many 
recent numerical and theoretical investigations (see e.g. 
[|i,[i|) and our findings add further support to it. 

We conjecture that the probability distribution P{5) 
of the of the rescaled variable 

S ^ if - fN)N^/VVTm (13) 

is of the form P{S) = exp(— G((5)) where the large S be- 
haviour of G{5) matches with the behaviour near the ori- 
gins of the functions L2(A/) and L{Af), i.e. 

G{S) ~ a_|(5|^/^ for(5^-c5o (14) 
G{6) ~ a+(5i2/5 for^^+oo (15) 

with a_ = 1.366 2] and a-|_ — .36. 

It is natural to assume that for a large class fRSB spin- 
glass model not only the large S behaviour of G{S) is in- 
dependent from of the model, but that the function G{5) 
does not depend on the model for all S and it is the same 
at all temperatures below the critical one.. A similar re- 
sults should be valid for replica symmetric models where 
the equivalent function H{{f — fN)N'^^^/q) is given by 
Tracy- Widom function [sl . 
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